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presented. Focus groups, interviews, long interviews, telephone 
interviews, and mail surveys are considered; and the advantages and 
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impact on survey research include sources of error, the nature of 
bias, and response rates. The code of ethics for survey research 
emphasizes that subjects should volunteer their participation, and 
the researcher should assure that no harm cones to them as a result 
of participation. Participants, who should be assured of anonymity 
and confidentiality if these are guaranteed, should know the purpose 
of the study and who the sponsors are. Respondents should be able to 
contact the researcher. Analysis and reporting should provide an 
honest accounting of the procedures, and unexpected findings should 
be acknowledged. "A Guide for Survey Research,*' a summary of 
advantages and disadvantages, is included. A chart summarizes 
comparisons of mail, f ace^to-f ace, and telephone surveys. (SLD) 
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A Critical Analysis of Interview, Telephone, 
and Mail Survey Designs 



Survey research is one of the more popular methodologies for 
dissertation projects. Survey designs have been described as 
"conversations with a purpose." Surveys require individuals to 
communicate with one another either in person, by telephone, or by 
written report (Bradburn and Sudman, 1988) . Both quantitative and 
qualitative research projects can incorporate survey research as a 
part of the methodology. This paper will present a critical 
analysis of current literature as it relates to personal 
interviews, telephone interviews, and mail questionnaires. 
Additional research concerns such as Source of Error, Bias, and 
Response Rate will also be addressed. The paper will conclude with 
a code of ethics for survey researchers. 



Interview Designs 

Focus Groups 

Focus groups can be useful in the beginning stages of a 
research project. The format allows the researcher to explore 
issues or problems with a small group of individuals. Researchers 
can generate questions or assumptions based on the informant's 
insights, evaluate different research populations, plan an 
interview schedule, and develop the questions for the survey 
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(Morgan, 1988) . Recently, one of my doctoral students used a focus 
group comprised of community college administrators to help her 
prioritize the issues to be addressed in her interviews. Her focus 
group was a small component of the larger research design which 
involved indepth interviews and a detailed questionnaire with the 
leaders of the community college system throughout the state. The 
use of focus groups linked to survey research is highly 
recommended • 

There are several advantages for using focus groups. It is a 
socially oriented research approach with an opportunity for people 
to interact with each other. In addition p it allows the researcher 
to probs issues, it has a high face validity, and is relatively low 
in cost (Krueger, 1988). On the limitations side, the interviewer 
has less control over the group as compared with an individual 
interview, data can be difficult to analyze out of context, the 
group can vary, and the interviewer should have some training 
(Krauger, 1988) . This technique has promise for assisting with 
survey research designs. 
Interviews 

Personal interviewing is the standard by which survey research 
gained credibility (Backstrom and Hursh-Cesar, 1981, p 19) . The 
personal interview is an interactive process between the researcher 
and the individual. There is a constant level of communication 
with a quality and quantity of information being exchanged. The 
flow of information will depend on interviewer, respondent, topic 
and situation. The communication process can become distorted or 
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biased at any time during the interview process (Warwick and 
Lininger, 1975) . There are several important considerations in 
selecting interviews as the research method- Utilization of the 
interview questionnaire will increase the chances of the subject 
participating in the study. A suggested minimal response rate for 
interviews is 75 %. The discussion mode increases the respondent's 
motivation for complete and accurate answers and allows the 
researcher to use more complex questions. It follows the 
researcher can also sequence the questions, clarify answers and 
have a longer questionnaire (Warwick and Lininger, 1975) . 

Babbie (1990) states that a properly designed interview will 
have an 80 to 85% completion rate. In addition to the completion 
rate, the advantage of an interviewer is a decrease in the number 
of "don't knows" and "no answers." The interviewer can make sure 
that the question is clear3.y understood and continue to probe for 
answers. Of note, is that the interviewer can observe the 
respondents and see their react ionj^: to the questions. 

Some of the disadvantage cited by Backstrom and Hursh-Cesar 
(1981) are interviewer biases, availability of the respondent, and 
high cost. This method is the most expensive in terms of cost and 
time. Yet, it can be a valuable resource for certain designs. A 
doctoral student wanted to chronicle the desegregation issue in the 
Denver Public Schools over a period of eighteen years. There was 
a great deal of historical information on the topic. As she worked 
on her methodology, she decided to add an interview component to 
her design. Over the course of a year, she interviewed seventy- 
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five individuals who were involved in leadership roles in the 
school district and community. The interviews were conducted in 
person, if possible, or by telephone if the respondent had moved. 
She reported complete cooperation from the respondents and they 
added a richness to her dissertation that was not possible with 
just written documents. 
The Long Interview 

Many qualitative research designs include a lengthy interview 
process. ^ith a limited number of subjects in the study, the 
researcher needs to have a longer questionnaire and a longer period 
of time. The questionnaire has several importan, 'unctions. First, 
it ensures that the investigator covers all the terrain in the same 
order for each of the respondents. Secondly, the researcher needs 
to have some distance from the respondents through certain 
questions or prompts. Lastly, the questionnaire provides the scope 
and course for the researcher (McCracken, 1988) . An example of a 
qualitative research design with a long interview process took 
place during a study trip. The researcher was the educational 
leader for a group of ten teachers on a two month trip to Egypt. 
The trip was financed by a grant written by the researcher. She 
decided to do her doctoral research by observing the group, 
rjCording detailed field notes, and conducting long interviews 
over an extended period of time. She wanted to explore the 
effectiveness of the program. Each participant in the project 
agreed to incorporate the cultural experience into their social 
studies curriculum during the following year. Most of the teachers 
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developed curriculum for dissemination, but there were a few 
participants who never used the experience in their classroom or 
completed ^he tasks associated with the project. 

0 

Telephone Interviews 

Telephone interviews are growing in usage. Backstrom and 
Hursh-Cesar (1981) suggested that telephoning is best for dealing 
with issues that influence or effect the general population, gives 
the researcher unlimited geographical access, and presents no 
physical risks to interviewers. As a research methodology, it is 
less expensive than personal interviews and provides greater 
accessibility to respondents in large cities. New computer 
technology has made it easier to use this as a research method. In 
comparison to personal interviews, there are several limitations. 
It is more difficult to establish rapport with the respondents over 
the telephone- There is no eye contact or body language to give 
the interviewer cues. The telephone interview tends to be shorter 
in length and the respondent could hang up at any time, thus ending 
the interview (Warwick and Lininger, 1975) . 

This method has become extremely popular for conducting polls 
and surveys. The Gallup Poll is an example of the telephone survey 
method used to compile the public's attitude regarding education. 
In September, the Kappan published the annual survey. Some of the 
topics addressed this year are "Biggest Problems Facing Local 
Public Schools," "Distribution of Condoms By Schools," and "Grading 
the Public Schools." 9% of the grades were A, 31% were B, and 33% 



were C, The authors noted that 40% of the respondents said the 
local schools were A's and B's, but only 18% gave the national 
schools the same grade. 

Mail Surveys 

The questionnaire is the physical representation of all the 
theories/ hypotheses, and hunches explored by the researcher. The 
problem with the defiiign is to create a set of questions and 
statements simple enough for the respondents to answer, but 
complex enough to develop a high quality research project 
(Backstrom and Hursh-Cesar, 1981) . 

The primary use of mail questionnaires is to conduct large 
surveys o' the general population where there is little risk of 
makiny a bad decision with the data. This method is also useful 
for working with well-defined populations and special interest 
groups- The researcher can gather extensive written data and 
demographic information. It is free of interviewer bias and poses 
no threat to the respondents. The main disadvantage is the 
traditional low return rate and the early returns are more biased, 
than the later responses (Backstrom and Hursh-Cesar , 1981) . One of 
the more extensive survey research projects was conducted by a 
doctoral student from Taiwan, First, she traced the Gross National 
Product of the Republic of China to see trends in employment 
over a period of ten years. Next she developed a survey 
questionnaire for a group of employers and a parallel form for 
employees in a wide range of occupations. The purpose of the 
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research was to revise the commercial education curriculum for the 
two year college system. The research design addressed current 
trends, educational experiences of the research population, and 
reconunendations for commercial educational skills needed for the 
future, 

A researcher must address the issues related to the 
reliability and validity of the literature used to generate the 
questions (Fowler , 1988) • In teaching survey research to doctoral 
students, we stress the importance of original sources, a through 
review of the literature, and a comprehensive analysis of prior 
research studies on the same topic. For survey research designs, we 
require Chapters 1, 2 and 3 to be completed before the survey is 
written. 

Another consideration which needs to be addressed is the 
variation in quality among surveys. Many researchers lack funding 
for their studies and therefore, many do not have an adequate stafi 
to assist with the project. There could also be a lack of 
knowledge about the methodological procedures. By not following 
high quality procedures, the researcher could affect the results of 
the study. The quality of the data will be no better than the most 
error prone feature of the survey design (Fowler, 1988) . 

Research Concerns 

Sources of Errors 

Fowler and Mangione (1990) describe a survey as a series of 
steps that together form a measurement process. The researcher 



needs to consider every aspect of the survey process for errors. 
The following table highlights some of the errors: 

A. The samples are a source of errors: 

1. When the sampling frame does not include everyone in 
the population to be described, thereby leaving out some types of 
people out of the sample; 

2. Because there is some probability that by chance alone 
a sample will not perfectly reflect the population from which it is 
drawn ; and 

3. When those selecx:ed to be in the sample do not provide 
answers either by refusing to participate altogether or by 
selectively refusing to provide answers to specific questions. 

B. Questions are sources of errors: 

1. When they are misunderstood; 

2. When they require information that the respondents do 
not have or cannot recall accurately; and 

3. When the respondents are not willing to answer 
accurately. 

C. Interviewers are a source of error: 

1. When they do not read the questions as worded; 

2. When they probe directively; 

3. When they bias answers by the way they relate to the 
respondents ; and 

4. When the record answers incorrectly. 

D. Data re(?'iction is a source of error: 



8 



1. When coders inconsistently apply coding rules or use 
faulty judgement about the appropriate codes to apply; and 

2 . When data are entered incorrectly into computer-usable 
files. (Fowler and Mangione, 1990, p 14) 

Nature of Bias 

The nature of bias will differ with personal interviews, 
telephone interviews, or mail questionnaires (Fowler, 1988). The 
personal interview is the most effective means of presenting the 
research. Yet, issues of availability and accessibility become a 
problem for the interviewer. Rural areas have a higher 
accessibility factor than cities. More telephone interviews are 
being utilized in the cities because of greater access to secure 
areas and calls can be scheduled at the respondents' convenience. 
The elderly and less educated individuals are less responsive to 
telephone contacts. 

In mail surveys, bias can be studied by comparing the early 
responders to those who responded later in the study. Fowler (1988) 
notes that the first 20% to 30% usually look different than the 
larger sample. The more educated individual will respond sooner 
than a less educated person. The best advice was to achieve a high 
response rate. 
Response Rate 

The response rate is largely under the control of the 
researcher (Fowler , 1988) . The question that arises next is: What 
is an adequate response rate? Babbie (1990) states that a response 
rate of 50% is generally considered adequate for analysis and 
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reporting. A response rate of 60% is considered good and 70% or 
higher is in the very good category. These are only guidelines and 
do not have any statistical basis. Backstrom and Hursh-Cesar 
(1981) gives a broader range for the response rates. They note 
that the Census Bureau expects a 99% return rate, but other forms 
of mail surveys will vary from 10% for the general population to as 
high as 80% for a well motivated group. 

Several studies have suggestions for increasing the response 
rate. A review of empirical studies by Kanuk and Berenson (1975) 
showed that follow-up letters and monetary incentives were the 
best techniques for increased response rates. Baumgartner and 
Haberlein (1984) found that salience had a powerful effect on the 
response rate. They also found that follow-up contacts by the 
researcher, incentives in the form of small amounts of money, a 
personal letter accompanying the questionnaire and the use of first 
class postage were helpful. Survey instruments sent via certified 
mail increased the response rate in the study conducted by Colton, 
Kane, Estes, and Elfoft (1990). A return rate of 96% was reported 
by Altschuld and Lower (1984) in a study which looked at 
principals' and teachers' perceptions and attitudes about t!ie 
evaliaation of teaching. They stated that the high return rate was 
due to careful planning of the survey process, support of the 
superintendent in each district, and the high degree of interest in 
the topic. 

Nonresponse is a possible source of error in the survey 
process. Nonresponse is not always biased, but it can be. Many 
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refusals to complete mail questionnaires is biased towards 
education and interest • Telephone nonresponse is related to age 
and education (Fowler, 1988) . The researcher should contact the 
nonrespop'lents to find out why they refused to participate in the 
study and include this information in the report. 

A Code of Ethics 
An analysis of the literature on personal interviews, 
telephone interviews, and mail questionnaires would be incomplete 
without addressing a code of ethics. This is a set of norms for the 
researcher to use as a guideline during the research process. In 
survey research, the subjects should volunteer their participation 
in the study. The researcher must assure that no harm comes to the 
respondent as a result of participation in the study. Most 
respondents prefer anonymity and confidentiality. The researcher 
should state this as part of the information shared with the 
respondents in the instructions or cover letter. The participants 
in the research project should know the purpose of the study and 
who are the sponsors. The respondents should be able to contact the 
researcher. The analysis and reporting must provide an honest 
accounting of the procedures. Sometimes findings are unexpected in 
survey research, and these need to be acknowledged. Ethical 
concerns should be addressed as an important part of survey 
research ( Babbie ,1990) . 
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A Summary of the Advantages 
and Disadvantages of Survey Research 



Potential advantages of personal interviewing: 

1. There are some sample designs that can be inplemented best by personal interview 
(e.g., area probability samples). 

2. Personal interview procedures are probably the most effective way of enlisting 
cooperation for most populations. 

3. Advantages of interviewer administration - answering respondent questions, probing 
for adequate answers, accurately following complex instructions or sequences - are 
realized, 

4. Multimethod data collection - including observations, visual cues, and self- 
administered sections - are feasible. 

5. Rapport and confidence building are possible (including any written reassurances that 
may be needed for reporting very sensitive material). 

6. Probably longer interviews can be done in person. 

Potential disadvantages of personal interviewing: 

1. It is likely to be more costly than the alternatives. 

2. A trained staff of interviewers that is geographically near the sample is needed. 

3. Total data collection period is likely to be longer than telephone procedures. 

4. Some samples (those in high-rise buildings or high-crime areas, elites, employees, 
students) may be more accessible by some other mode. 

Potential advantages of telephone interviewing: 

1. Lower costs (compare personal interviews.) 

2. Random-digit dialing sampling of general populations. 

3. Access to certain populations (compare especially personal interview). 

4. Potential for short data collection period. 

5. The advantages of interviewer administration (compare mail). 

6. Interviewer staffing and management easier (compare personal interviews) - smaller 
staff needed, need not be near sample, supervision and quality control potentially 
better. 

7. Likely better response rate from a list sample (compare mail only). 

Potential disadvantages of telephone studies: 

L Sampling limitations, especially problem of omitting those without telephones. 

2. Nonresponse associated with RDD sampling (compare personal). 

3. Questionnaire or measurement constraints, including limits on response alternatives, 
impossibility of visual aids, and interviewer observations. 

4. Possibly less appropriate for personal or sensitive questions if no prior contact. 

Potential advantages of self-administered (compare interviewer-administered) data collections: 

1. Ease of presenting questions requiring visual aids (compare telephone). 

2. Asking questions with long or complex response categories. 

3. Asking batteries of similar questions. 

4. The fact that the respondent does not have to share answers with an interviewer. 
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Potential disadvantages of self-administration: 

1. Especially careful questionnaire design is needed. 

2. Open questions usually are not useful. 

3. Good reading and writing skills by respondents . e needed. 

4. The interviewer is not present to exercise quality control with respect to answering 
all questions, meeting question objectives, or the quality of answers provided. 

Self-administered surveys can be done by mail, via group administration, or in households. 
Each approach has strengths and potential weaknesses. 

The advantages of group administration: 

1. Generally high cooperation rates. 

2. The chance to explain the study and answer questions about questionnaire (compare 
mail). 

3. Generally low cost. 

The main disadvantage is that only a small number of surveys can use samples that can be 
gotten together in a group. 

The advantages of mail procedures: 

1. Relatively low cost. 

2. Can be accomplished with minimal staff and facilities, 

3. Provides access to widely dispersed samples and samples that are difficult to reach 
by telephone or in person for other reasons. 

4. Respondents have time to give thoughtful answers, to look up records, or consult with 
others. 

Disadvantages of mail procedures: 

1. Ineffectiveness of mail as way of enlisting cooperation (depending on group to be 
studied). 

2. Various disadvantages of not having interviewer involved in data collections. 

3. Need for good mailing addresses for sample. 

Dropping off (and later picking up) a questionnaire at a household has advantages: 

1. The interviewer can explain the study, answer questions, and designate a household 
respondent (compare mail). 

2. Response rates tend to be like those to personal interview studies. 

3. There is opportunity to give thoughtful answers, consult records or other family 
members (compare personal or telephone). 

The disadvantages include the following: 

1. This procedure costs about as much as personal interview. 

2. A field staff is required (albeit perhaps a less thoroughly trained one than would be 
needed for personal interviews). 

From: Fowler, F.J. Jr. (1988). Survey Research Methods . Newbury Park, CA: Sage Publications. 
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Survey Research 



Comparison of Mail, Face to Face, and Telephone Survey Methods 



Factor 


Mail 


Face to 
Face 


Telephone 


1. 


Cost 


1 


4 


2 


2. 


Personnel requirements: interviewers 


N/A 


4 


3 


3. 


Personnel requirements: supervisors 


2 


3 


4 


4. 


Implementation time 


4 


4 


1 


5. 


Sample coverate 


3 


1 


1 


6. 


Response rate-general public 


4 


2 


2 


/. 


Refusal rate 


Unknown 


3 


3 


Q 

o. 


Noncontact/nonaccessibility 


2 


3 


2 


o 

y. 


Ability to obtain response from an elite 


4 


1 


2 


10. 


Respondent within household 


4 


2 


2 


11, 


Interviewer control 


N/A 


3 


1 


12. 


Socially desirable response 


1 


4 


3 


13. 


Item nonresponse 


3 


2 


3 


14. 


Length of questionnaire-impact or response 


3 


1 


2 


15. 


Confideniiality 


4 


4 


4 


16. 


Ability to ask sensitive questions 


2 




2 


17. 


Ability to probe 


4 




2 


18. 


Ability to clarify 


4 




2 


19. 


Ask complex questions 


3 




3 


20. 


Use of open-ended questions 


4 




1 


21. 


Use of visual aids 


2 




4 


22. 


Avoid opportunity for consultation by respondent 
with others 


4 




1 



Key: 1 = Major advantage 3 = Minor disadvantage 

2 = Minor advantage 4 = Major disadvantage 



From: Frey, J. H. (1983). Survey Research bv Telephone . Beverly Hills, CA: Sage Publications. 
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